ATTORNEY'S DOCKET 
4904.2-7 



1 



PATENT APPLICATION 



METHOD AND SYSTEM FOR MEASUREMENT OF THE DURATION AN 
AREA IS INCLUDED IN AN IMAGE STREAM 

TECHNICAL FIELD OF THE INVENTION 

This invention relates in general to the field of video transmission systems 
and, in particular, to a system and method for measuring the duration an image is 
included within an image stream. 



ATTORNEY'S DOCKET PATENT APPLICATION 

4904.2-7 



BACKGROUND OF THE INVENTION 

Teams, leagues, and venue owners typically charge advertisers increased fees 
when their banners or signs appear periodically during portions of a sports telecast. 
Sporting events have long provided for additional advertising revenue generation 

5 during a telecast event as well as during commercial breaks. For example, 
broadcasters charge a fee per on-air time to advertisers having commercials viewed 
during portions of a sporting event telecast. Advertisers are often charged fees related 
to the on-air time in the telecast that viewers may see advertisement logos and banners 
common in sporting arenas and parks. For example, the walls of a baseball field are 

10 commonly comprised of numerous adjacent panels, one or more of which are typically 
leased to advertisers for display of an advertiser logo. Ice hockey also includes 
advertising banners typically displayed on the dasher boards of the hockey rink as well 
as beneath the ice itself. In addition, football fields often include sponsor logos 
painted on the football field in addition to logos on the stadium walls. Sporting 

15 franchises and sporting broadcasters also use rotating banner advertisements. These 
banner advertisements, rather than being fixed for the duration of a broadcast, are 
maintained on a rotatable chassis that allows different wall panel advertisements to be 
displayed during the game. A broadcaster may be interested in measuring portions of 
images that are displayed during a broadcast of the video. 

20 One incentive for inserting images into video is for realizing enhanced or 

additional advertising revenue for television broadcasters. Many systems and 
methods have been proposed for inserting static and dynamic images, such as 
advertising, into video in real time. These prior art systems and methods suffer from 
various drawbacks and problems, many of which are detailed in US Patent No. 

25 5,892,554 to DiCicco, et al A newly developed technique promises to revolutionize 
advertising in, for example, sporting event broadcasts. Computer facilitated broadcast 
systems are able to insert images, for example a graphic image of an advertiser's 
banner, into a video stream telecast. Generally, a site from which a telecast is to be 
performed has a three-dimensional model made thereof. The model includes target 

30 areas for insertion of graphic or video images. The target areas may be real areas of 
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the site, for example a dasher board of a hockey rink, or may be imaginary surfaces, 
for example synthetic billboards. Generally, each frame of the video transmission is 
analyzed and images written into the target areas in any frames containing the target 
areas being broadcast. A three-dimensional model of the site allows for realism to be 

5 obtained during various camera orientations and focus. 

Broadcasters must be able to determine the on-air duration of advertisements 
physically present or inserted into the video stream. Unfortunately, techniques for 
measuring an exposure time for these images during a telecast suffer from many 
disadvantages. For example, the duration of the broadcast of physical signage is 

10 calculated manually, and typically only after the broadcast has been performed. 
Manual calculation of the telecast transmission time, or on-air time, of an 
advertisement is generally performed by an operator, using accrued measurement 
techniques such as a hand-actuated stopwatch. This method is subject to a multitude 
of errors and inaccuracies, including failures in operator attentiveness and operator 

15 delay time. Moreover, the operator performs these measurements by watching a tape 
of the original broadcast sometime after the broadcast has been made. These 
techniques also require multiple operators to watch the same or another version of the 
taped broadcast to make measurements for multiple advertisements. As a result, these 
traditional methods require large numbers of resources and suffer from measurement 

20 inaccuracies and delays which in turn limit their use. 

Moreover, tariffs for advertisements are usually calculated using these prior art 
techniques and are typically limited to revenues that may be derived from traditional 
and existing banner advertisements. 

The terms 'telecast' and 'broadcast' are used within the description to 
25 facilitate understanding of the invention. However, telecast and broadcast media are 
exemplary only. The present invention is not limited to application to television 
transmissions. Rather, the present invention may be used in any number of 
communication technologies, such as television, cable, Internet, satellite, digital 
theater, etc., capable of transmitting graphic media (e.g., image streams). 
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While the technology for providing inserted images into a video is being 
developed, techniques for accurately assessing advertising revenues therefrom have 
yet to be addressed. To implement a fair fee charging system, the duration of the 
broadcast of these banners is calculated manually and typically after the broadcast has 
5 been performed. Manual calculation of the telecast transmission time, or on-air time, 
of an advertisement is generally made by a person watching a tape of the original 
broadcast some time after the broadcast has been made. Often, calculation of the on- 
time transmission of a particular advertisement is made by such crude and error prone 
techniques as a simple had actuated stop-watch. Heretofore, tariffs for advertisements 
10 made according to image inserting techniques are limited to the prior art techniques 
utilized for traditional banner advertisers. 
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SUMMARY OF THE INVENTION 

An object of the present invention is to provide a method and system for 
calculating the duration synthetic and/or real images, or portions thereof, are included 
in an image stream which overcome one or more problems with prior art methods and 
5 apparatus. 

In accordance with an embodiment of the present invention, a system and 
method for calculating the duration one or more target areas is included in an image 
stream utilizes a three-dimensional model of a site including at least one target area 
assigned to a physical area within the site from which the image stream is captured. A 
10 counter maintained in association with the target area is incremented during periods, 
for example on a frame-by-frame, or field-by-field, basis, when the target area is 
included in the image stream. Accordingly, the counter maintained in association 
with the target area provides an indication of the duration that an image within the 
target area is included in an image stream. Thus, the counter provides a duration 
1 5 measurement that the portion of the site included in the physical area corresponding to 
the target area (an image captured therefrom referred to as a physical image) is 
included in the image stream. 

In accordance with another embodiment of the invention, the image stream 
captured by an image capture system, such as one or more cameras, may be processed 
20 to include synthetic images not present in the site from which the image capture was 
performed. The captured image stream may be manipulated by image insertion 
technologies and processed to include the synthetic images inserted into a target area 
associated with a physical area of the site. Accordingly, the counter provides an 
indication of the duration a synthetic image is included in the target area of the 
25 processed image stream. 

In accordance with another embodiment of the present invention, the 
percentage of the target area within the field of view may be used to determine 
whether or not a counter associated with the target area is incremented. 
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In accordance with another embodiment of the present invention, an occlusion 
parameter associated with a target area is calculated for adjusting the calculated 
duration the target area is included in the image stream. The occlusion parameter 
facilitates accounting for variations in the obstruction to an image (physical or 
5 synthetic) corresponding to a position of the target area and is used for making 
adjustments to a counter so that the count thereof is indicative of variations in 
obstructions of the image. 

In yet another embodiment of the present invention, a foreground parameter 
associated with a target area is calculated for adjusting the calculated duration the 
10 target area is included in the image stream. The foreground parameter facilitates 
accounting for variations in the viewability of an image within the target area that 
results from various camera zooms. 

In yet another embodiment, the counter can be triggered by a threshold of the 
percentage of the visible video image covered by the target area, both considering and 
1 5 excluding the area of any occluding bodies. 

In yet another embodiment of the invention, real-time calculations of the 
duration that one or more target areas are included within an image stream provide a 
mechanism for ensuring a desired duration a synthetic image is included within an 
image stream is met. Analysis of metrics or other statistical data collected during 
20 image capture or transmission and indicative of the duration a target area is included 
within an image stream may indicate that image data assigned to a particular target 
area is unlikely to be included within the image stream for a predefined, desired 
duration. The image data may then be reassigned to another target area in an attempt 
to achieve the desired duration of inclusion of the image data within the image stream. 

25 These and additional objects and advantages of the invention will be apparent 

from the following description of a preferred embodiment of the invention, made with 
reference to the appended drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention, the objects and 
advantages thereof, reference is now made to the following descriptions taken in 
connection with the accompanying drawings in which: 
5 FIGURE 1 is a schematic representation of a real time image insertion system 

for manipulation of image streams; 

FIGURE 2 is an example of a video image generated by a video camera prior 
to insertion of a target image; 

FIGURE 3 is a rendering of a model of a site at which the video image of 
10 FIGURE 2 was taken and in which is defined a target area containing a reference 
image; 

FIGURE 4 is an image containing a rendering of the model of the site with the 
reference image, the image rendered from the same position and angle of the camera 
generating the video image of FIGURE 2; 
15 FIGURE 5 is a target area processing mask generated from the image of 

FIGURE 4; 

FIGURE 6 is a masked reference image generated by applying the mask of 
FIGURE 5 to the image of FIGURE 4; 

FIGURE 7 is a masked background image generated by applying the target 
20 area mask of FIGURE 5 to the original video image of FIGURE 2; 

FIGURE 8 is a masked target area image generated by applying the target area 
mask of FIGURE 5 to the original video image of FIGURE 2; 

FIGURE 9 is an occlusion image generated by comparing the masked target 
area image of FIGURE 8 to the masked reference image of FIGURE 6; 

25 FIGURE 10 is a final, composite image, containing a target image that is 

generated by combining the occlusion image of FIGURE 7 with the masked 
background image and the masked target image; 
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FIGURE 1 1 is a schematic representation of an image measurement system 
where the measurement process for a video stream is implemented in a multi-camera 
system before a production system which chooses one or more of the multiple input 
signals as output; 

5 FIGURE 12 is a flowchart of the processing of a module for calculating the 

duration an image is inserted into a video telecast; 

FIGURE 13 is a final image illustrating the scenario depicted in FIGURE 8 
after an elapsed period of time; 

FIGURE 14 is a flowchart depicting the processing of a module operable to 
10 adjust the calculation of the duration that an image is inserted into a video to reflect 
variations in occlusions of the inserted image; 

FIGURES 15A and 15B respectively illustrate a final, composite image 
obtained at different camera positions, pan and tilt angles, and zoom settings; 

FIGURE 16 is a flowchart depicting the processing of a module operable to 
15 adjust the calculation of the duration that an image is inserted into a video to reflect 
variations in viewability of an inserted image resulting from differences in camera 
zoom; 

FIGURE 17 illustrates a target area having multiple sub-target areas according 
to an embodiment of the present invention; and 
20 FIGURE 18 is an image frame having various image areas designated for 

specifying thresholds for allowing/disallowing increments to a target area counter. 
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DETAILED DESCRIPTION OF THE DRAWINGS 

The preferred embodiment of the present invention and its advantages are best 
understood by referring to FIGURES 1 through 18 of the drawings, like numerals 
being used for like and corresponding parts of the various drawings. 

5 One application for a real time image insertion system for manipulation of 

image streams is in a live broadcast of a sporting or entertainment event or the like 
from a stadium, arena, track, course or other sports or entertainment venue. Such a 
system is described below in connection with this application. The system of the 
present invention may be used for real-time measurement of exposure duration of 

10 synthetic images in an image stream. For example, the duration inserted images, such 
as advertisements that are not physically present in an entertainment venue from 
which an image stream is captured, manipulated and broadcast, may be accurately 
measured in real time according to the teachings of the invention. Advertising 
revenues and/or fees for these inserted images may then be calculated as desired in 

15 response to the calculated duration the inserted images are included within the image 
stream. 

Although the image duration measurement system of the present invention has 
particular advantages when used in conjunction with an image stream manipulation 
system, it can also be used in other applications. The system of the present invention 
20 may also be used for a real-time measurement of the duration an image is captured 
from a physical object present in the site from which the image stream is captured, 
such as images captured from a physical advertisement in an entertainment venue 
from which an image stream is captured. 

The details of image measurement, and the techniques of manipulating image 
25 streams, as performed by the image measurement system 100 illustrated in FIGURE 1 
are outside the scope of the present invention and the technique described is 
illustrative only. It should be understood that the image insertion system is operable 
to manipulate a three-dimensional model of a site from which an image stream is 
captured and insert images, for example bit maps providing graphic depictions of 
30 advertisements, into frames of the image stream. Thus, an image stream may include 
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images obtained from an image capture system, for example a camera system 104A- 
104N, as well as images of the image stream having synthetic image/s inserted 
therein, an exemplary technique including operations generally described below with 
reference to FIGUREs 1 through 10. 
5 A captured image stream is defined herein as one or more image frames (or ? 

alternatively, fields) captured from physical objects present in the site by an image 
capture system. An image stream is defined herein to refer to the captured image 
stream and/or the product of processing that may be performed on the captured image 
stream such as manipulation of the images within the image stream including 
10 manipulation by image insertion technologies. The image stream may include images 
solely captured from the physical site from which the image stream is captured, or the 
image stream may include images captured from the site and/or synthetic images 
(inserted into the image stream) not physically present within the site from which the 
image stream is captured. 

15 A physical image is defined herein to refer to an image in an image stream that 

is captured from a physical object with the image capture system. A synthetic image 
is defined herein to refer to an image inserted into an image stream that is not captured 
from an object physically present in the site from which the image stream is captured. 

A target area is defined herein to refer to an area of interest within a physical 
20 site and/or a corresponding area within a model of the physical site. A captured image 
stream and an image stream are said to include the target area, or a portion thereof, 
when the field of view of an image capture system includes the area of interest (or a 
portion thereof). For example, a dasher board in a hockey arena may be specified as a 
target area. The dasher board specified as the target area will have a corresponding 
25 area in a three-dimensional model of the hockey arena that is referred to as the target 
area as well. The present invention contemplates the calculation of a duration that one 
or more target areas may be included within an image stream obtained by an image 
capture system. The target areas may be present in an image stream that may be 
obtained during a single event in a particular venue, or multiple events in separate 
30 venues. 
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A target image is defined herein as an image within the target area in an image 
stream and may be a physical image or a synthetic image or a composite image 
thereof. 

A reference image is the appearance of a target area surface within the actual 
5 site that will be used for occlusion processing and may be an image associated with a 
target area that is rendered by an image rendering system. The reference image may 
be a rendered image of a physical object (such as a physical object within the site from 
which the image stream is captured), or the reference image may be a rendered image 
of a synthetic image that is to be inserted into a target area of an image in an image 
10 stream. Reference images are, in general, utilized to form a target image for either 
insertion into a target area and facilitation of duration measurement thereof or to 
facilitate duration measurements of physical images having target areas assigned 
thereto. Synthetic target images are developed, in part, by a rendered reference image. 

As described herein, the present invention provides a technique and 
15 mechanism for calculating the duration the target area is included in an image stream. 
Accordingly, the technique and mechanism provide for calculating the duration an 
image (physical or synthetic) is included within the target area may be applied to 
calculating durations an image is included within an image stream. 

Referring to FIGURE 1, a real time image measurement system 100 for 
20 calculating the duration one or more target areas are included in image streams is 
schematically represented by its primary functional components. These components 
are implemented as a combination of hardware and software, and are not intended to 
represent discrete hardware or software components or as being limited to any 
particular implementation unless otherwise noted. 

25 The image measurement system 100 receives an image stream from an image 

production system 102. The image production system 102 selects an image signal 
from a camera system 104 A or one of a plurality of other camera systems 104B-104N. 
This selected image stream is then provided to an image measurement system 100 for 
duration measurements of physical images within the physical venue, such as existing 
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signage, synthetic images such as advertising, or areas within the image stream that 
may have synthetic images inserted therein after the image stream is subjected to 
processing. The image measurement system may be located with the video 
production system at, for example, a mobile production facility. It may also be 
5 remotely located at a central production facility or even further downstream, such as at 
a local television station or cable operator. Alternately, image duration measurements 
may take place before selection of an image stream, for example by measuring images 
in the image stream from each camera system prior to it being provided to the 
production system. 

10 In another embodiment, production system 102 may incorporate further 

functionality into the image measurement system as will be discussed in further detail 
< in conjunction with FIGURE 1 1 , Generally, the output of production system 102 with 
multiple inputs provided by other camera systems 104B-104N may be compared and a 
duration counter for a camera input selected by a production control unit such as a 

1 5 production truck may be updated. For example, multiple cameras are commonly used 
to capture image streams from a single venue. Image streams from the multiple 
cameras are then provided to a production system. The production system is operable 
to provide an input to an image stream/telemetry separator 118, Generally, the input 
at any given instance provided to image stream/telemetry separator 118 will include 

20 an image stream taken from a single camera of the other camera systems 104B-104N. 
Thus, production system 102 is operable to switch the image stream that is provided 
to video/telemetry separator that is processed and ultimately output a selected image 
stream from the system in the form of a video telecast or other processed image 
stream. Accordingly, because a counter (as described more fully hereinbelow) is used 

25 to accumulate a measure of the duration that an image is included in an output image 
stream, a switching mechanism must be utilized such that the counter is incremented 
only when an image is included in an image stream being output by the production 
system 102 rather than incrementing the counter when the image is included in any 
image stream captured by one or more of the other camera systems 104B-104N. 



ATTORNEY'S DOCKET 
4904.2-7 



13 



PATENT APPLICATION 



A camera system 1 04A generates an image stream encoded with telemetry data 
indicating at least where the camera is pointing in relation to a known or predefined 
three-dimensional frame of reference of a site. The camera system includes a 
conventional video camera 108 connected to a camera head 110. The camera head 110 
5 includes sensors that generate information indicating the azimuth and elevation, or 
some other coordinates defining the direction of the optical axis of the camera 108. 
This telemetry information is provided to the telemetry processor and encoder 112, 
Camera 108, or some other attached sensors, also provides the telemetry processor 
and encoder with additional telemetric information indicating, for example, the focal 

10 length and aperture of the camera's lens. The focal length will vary with the degree of 
zoom of the lens. The aperture will vary with changing light conditions. Optionally, 
additional sensor information, for example global positioning satellite information 
114 may also provide information to the telemetry processing and encoder indicating 
the position of the camera in terms of its longitude, latitude and elevation. The 

15 position of the camera can easily be determined and may be permitted to move rather 
than remain fixed in a predefined location. Using an image stream timing signal 
provided by the camera, the telemetry processor and encoder generates a data signal 
that can be synchronized with the image stream generated by the camera. This data 
signal encodes the telemetric information for each frame of the image stream 

20 generated by the camera. An image stream/telemetry combiner 116, which may be 
part of the telemetry processor, then combines the data signal with the image stream. 
With the telemetry information synchronously encoded in the image stream, sufficient 
information is provided to allow synthetic images to be inserted into the image stream 
at any point downstream of the camera system. 

25 While manipulation of an image stream is described herein with references to 

one or more frames that are included within the image stream, it should be understood 
that the frames may be comprised of sub-elements, for example fields. For example, 
frames of a typical video telecast are comprised of even and odd fields. Thus an 
image stream comprising a video stream or telecast having a 30Hz frame rate may 

30 actually be formed by two 60Hz fields. Manipulation and analysis of an image stream 
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or image telecast described herein with reference to frame-based operations is 
illustrative only and is described as such for simplification of discussion. It is 
understood that the same general procedures and techniques may be implemented by 
using field-based operations. 

5 Once image measurement system 100 receives an encoded image stream, 

image stream/telemetry separator 118 extracts the telemetry data for a particular image 
within the image stream. The image stream is further decoded by an image stream 
decoder/buffer 1 19 to extract and store an image from each frame of the image stream. 
An example of an image included in an image stream generated by a camera is 

10 illustrated as image 400 in FIGURE 2. This particular example is of an ice hockey 
game. It includes a dasher board 402, a first hockey player 404 and a second hockey 
player 406. The operation of the image measurement system 100 and the image 
measurement process will be described below in reference to image 400, However, 
the image measurement process will be repeated for an image in each successive 

15 frame of an image stream, at least to the extent the image changes between frames. 

A controller 120 represents a software and hardware entity, or a collection of 
entities, that coordinate processes occurring within the functional components of 
image measurement system 100. Using the telemetry data and other information 
describing an event occurring at the site, for example the inning number of a baseball 

20 game, the score or other information on the flow of a sports game, controller 120 
accesses predefined image measurement rules in database 122 to determine, based at 
least in part on a camera identifier embedded in the telemetry data, what areas - 
referred to herein as target areas — included in the venue and that may be included in 
the image stream are to be subjected to a duration measurement throughout the image 

25 stream (or a portion thereof). The target area may be, for example, an area within a 
sporting arena in which advertising displays will be synthetically inserted within the 
processed image stream. Alternatively, the target area may be, for example, an area 
within the sporting arena in which physical signage is present at the site and may be 
captured by the image stream. Thus, a target area of the site that is included in an 

30 image stream may have physical image/s and/or synthetic image/s included therein. 
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Having predefined rules allows a pre- selected target area to be subjected to a duration 
measurement that results in a quantitative measure of the duration the target area is 
included in an image stream. By calculating a quantitative measure of the duration a 
target area is included within an image stream, the duration that a physical image or 
5 synthetic image included within the target area is also made. Furthermore, a target 
area may change at predefined times or periods, or based on the status of the event 
being subjected to image capture, that is the location of a target area in a site may 
change during image stream capture or processing. 

An advertising director, for example, may also monitor and control 
10 measurement processes during an event using a director's console 123 that includes 
software and hardware for inputting commands and data to controller 120, For 
example, the director may provide the system with information concerning the state of 
the event at the site, if such information is not otherwise available from a database. 
The director may also override the image measurement rules in database 122 and 
15 manually select target areas for duration measurements, or the director may modify 
the rules dynamically. The advertising director may also set up and maintain 
databases that maintain the target areas. The advertising director's console will 
include monitors so that the director can monitor the image stream during duration 
measurements of target areas. The director's console may also allow the director to 
20 modify databases storing CAD models for the reference images, which are described 
below, and to monitor and adjust steps of the target area measurement process as 
described below. 

For each target area included within the image stream or processed image 
stream, a reference image may be assigned to a predefined target area at the site. The 

25 reference image is rendered based on a predefined reference model of the target area. 
More than one target area may be defined and appear in any given image stream. The 
model is, preferably, a computer aided design (CAD) model that defines surfaces (real 
or imaginary) of target areas mathematically thus allowing the model to be rendered 
from any angle. The telemetry data extracted from the captured image stream allows 

30 the model to be rendered from a viewpoint that is substantially the same as the 
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location and orientation of the camera generating the image stream. This rendering, 
which is in effect a synthesized image of the target areas of the site that is aligned with 
the image stream, is then used to guide duration measurements of the target areas 
within the image stream. If the camera position changes between frames, this 
5 rendering takes place for each such frame. However, if it does not change between 
frames, the renderings for the prior frame may be used. Accordingly, a reference 
image corresponding to a physical image object may be rendered at the various 
orientations and focus from which the physical image may be captured. 

Referring briefly also to FIGS. 2 and 3, image 502 of FIGURE 3 is an example 

10 of a rendering of a predefined model of the site, in which the image shown in 
FIGURE 2 was taken. A computer aided design (CAD) system, which can be 
standard, commercially available CAD software executing on a computer, generates 
the rendering from the predefined model. Note that the rendering is not made from the 
same position and camera angle as the video image of FIGURE 2. A target area 504 

15 in this example is a predefined area of the surface of the dasher board 402. The model 
could also have imaginary surfaces defined as target areas. For example, the model 
could have defined the position of an imaginary banner hung from the ceiling of the 
hockey arena. A target area surface, real or imaginary, need not be flat. In this figure, 
the target area has been rendered with a reference image. The reference image can be 

20 stored, for example, as a bit map image that is inserted into the rendered model. In this 
particular example, it is a blank white wall, that is the portion of the dasher board 
having a target area designated therewith is white, or blank, and is accordingly 
represented by a like reference image. However, the reference image may be an image 
of an advertisement that is physically located on, for example, dasher board 402 such 

25 as a corporate logo. The reference image may be rendered from various angles to 
correspond to the different orientations that the physical area having the target area 
assigned thereto may be captured in the image stream. The reference image may be an 
image of a synthetic element to be later inserted into the image as well. For example, 
an advertisement may be inserted within a specific target area and thus appear in the 
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target area of a final processed image. The reference model of the site with reference 
images for each target area is stored in a CAD file 124 shown in FIGURE 1 . 

Referring now to FIGURES 1, 2 and 5, CAD model renderer 128 renders a 
baseline image 700 of the CAD model stored in CAD file 124, based on the telemetry 
5 data from the camera that has captured image 400 shown in FIGURE 2. Baseline 
image 700 of FIGURE 4 includes target area reference image 506 inserted into target 
area 504. As previously described, the telemetry data indicates the identification, 
angle, focal distance and aperture setting of the camera taking the image 400. The 
telemetry data may also indicate the camera's location if the camera it not fixed in 
10 location. 

The image duration measurement process of the present invention may also 
include generating a target area mask. FIGURE 5 illustrates mask image 900 for the 
exemplary image 400 of FIGURE 2. A target area mask 502 is generated by a mask 
builder 134 using the reference image generated by model renderer 128. To generate 
15 the mask image 900, target area mask 502 within mask image 900 has all pixels 
therein set to a predefined value and all pixels outside target area mask 502 are set to 
another predefined value. In the mask image 900, all target area mask 502 pixels are 
white and all pixels surrounding target area mask 502 are black. 

Target area mask 502 is used for several purposes in connection with an 
20 occlusion processing step. Referring now to FIGURE 1, target area mask 502 is used 
by a background/target area reference image separator 138 to separate or mask each 
target area reference image 506 within baseline image 700 generated by model 
renderer 128. In the example illustrated in FIGURE 4, the mask will be used to 
separate target area reference image 506 within baseline image 700 from the rest of 
25 the image, resulting in a masked reference image 700a shown in FIGURE 6. Target 
area mask 502 is used to locate and separate within the original image the target area/s 
from the rest of the non-target areas of the image, which will be referred to as the 
background. This function is performed by background/target area separator 136. 
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FIGURE 7 and FIGURE 8 illustrate the two images that result from separating 
the background from the target area in the original image 400 shown in FIGURE 2. 
FIGURE 7 is a masked background image 400a, which includes all of the original 
image except that portion within target area 504, which is blank. FIGURE 8 is a 
5 masked target area image 400b, which includes a portion 1200 of the original image 
400 included within target area 504. The mask is also used by background/target 
image separator 140 to separate the target images within the image rendered by model 
Tenderer 130. 

In the example image 400 of FIGURE 2, part of hockey player 404 is covering 

10 a portion of dasher board 402 having target area 504 assigned thereto that is to be 
subjected to duration measurements according to the invention. In order to measure 
the duration target area 504 is included in an image stream including image 400 and to 
provide a quantitative measure of the duration that reflects the "viewability" of an 
image included within target area 504, the portion of hockey player 404 within target 

15 area 504, which is referred to as an occlusion, must be differentiated from the rest of 
target area 504 of original image 400, and the corresponding area removed from the 
remaining, unoccluded image within target area 504. To make this separation, 
occlusion separator 144 compares masked target area image 400b to masked reference 
image 700a. Any differences are presumed to be occlusions, i.e. images of objects 

20 between the camera and the surface (or synthetic surface) having target area 504 
assigned thereto. Small differences between the masked reference image and the 
masked target image introduced by electronic noise in the camera may be 
accommodated using a number of techniques commonly practiced in the field of 
image processing, for example small region suppression. Imprecise positioning of the 

25 target area due to errors in the telemetry measurement system may be accommodated 
by filtering the telemetry data using commonly practiced target tracking techniques, 
for example Kalman filtering. In the illustrated example, masked target area image 
400b, shown in FIGURE 8, is compared to masked reference image 700a, shown in 
FIGURE 6. The resulting occlusion image 400c shown in FIGURE 9 includes only the 

30 occlusion which, in this illustrative example, is the portion/s of hockey player 404 
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captured in image 400 within target area 504. The rest of occlusion image 400c is 
blank. The occlusion separator 144 also creates an occlusion mask. The occlusion 
mask identifies the portions within the original image 400 that constitute the 
occlusion/s. In the example, the occlusion mask is generated from occlusion image 
5 400c of FIGURE 9. Masked background image 400a, masked target image 800a and 
occlusion image 400c may be combined by image combiner 146 to form a composite, 
or final, image 400e that may be inserted into the image stream in place of image 400. 
Target image 604 may include a synthetic image and, accordingly, the composite 
image generated provides a manipulated image including the synthetic image having 

10 an occlusion such that a portion of the synthetic image is included within the final 
image and appears as if the synthetic image were part of original image 400. The 
masked background image, masked target image and occlusion image are combined 
by the image combiner 146 to form a composite, or final, image. The final image may 
be used by image manipulation technologies for insertion into the image stream. 

15 Accordingly, the final image, or constituent components thereof, may be used by the 
image measurement system 100 for duration calculations of target areas included 
therein. The final image may then be inserted into the image stream when the 
reference image included synthetic images. However, the final image, or portions 
thereof, may be used only for calculating duration measurements associated with a 

20 target area and may not be inserted into the image stream. The latter scenario may be 
used to calculate duration measurements for physical signage having a target area 
assigned thereto in which case the final image composition facilitates a duration 
calculation of image data corresponding to a physical image associated with the target 
area while the image stream is not subjected to image insertion technologies. 

25 Accordingly, calculation of a duration measurement provides an indication of the 
duration that physical image/s are included within the image stream. 

In the most simple form, a system of determining a duration measurement that 
target area 504 is included within an image stream includes counting a number of final 
images 400e in an image stream that include target area 504. The cumulative duration 
30 of a portion of an image stream having target area 504 and an image included therein 
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may then be calculated based on, for example, the frames per second in the image 
stream transmission and/or capture. Thus, the system for measuring the duration that 
one or more target areas, and thus the duration image data is encompassed therein, is 
included in the image stream produced by the image capture system and/or image 
5 manipulation system includes analyzing the final images 400e, or constituent 
components thereof, for the inclusion of one or more target areas 504 and, by 
association with the target area, the corresponding inclusion of image data 
encompassed within target area 504. 

With reference now to FIGURE 11, there is illustrated an image capture and 

10 target area duration measurement system according to the teachings of the invention. 
One or more image capture and telemetry systems 104A-104N may capture an image 
stream from a site such as a sporting or entertainment venue. Each image capture and 
telemetry system 104A-104N includes a respective telemetry device. Accordingly, 
image data captured by an image capture and telemetry system 104A-104N has 

15 telemetry information encoded within the captured image stream. The telemetry 
information may include image capture device positioning data, image capture device 
orientation data, focal information related to the zoom of the image capture device, 
etc. Image and telemetry data may be output from each image capture and telemetry 
system 104A-104N to one or more image duration measurement systems 100A-100N 

20 and to a production system 102. Production system 102 may perform processing of 
captured image data and provides switching functions for selecting one of image 
capture and telemetry system 104A-104N image data to be output. Typically, an 
image stream output will include an image stream, or processed image stream, from a 
single image source at any given moment. To implement a duration measurement that 

25 is attractive to advertisers, the present invention provides for accurately measuring the 
duration a target area is included in an output image stream. 

Because numerous image capture and telemetry systems 104A-104N may be 
used to supply production system 102 with various image views of an event, different 
views may include a particular target area while others exclude the target area. 
30 Accordingly, each image capture and telemetry system preferably embeds a tag that 
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identifies the corresponding camera in the image and telemetry data captured thereby. 
Other techniques for identifying the camera and/or image and telemetry data are 
known. Accordingly, production system 102 can increment a counter that measures 
the duration a target area is included within an output image stream. Coordination of 
5 such measurements may be performed by a measurement coordination device or 
system 121 A that is supplied with image measurement data obtained from image 
capture and telemetry systems 100A-100N as well as an output image stream. 
Accordingly, a counter assigned to a target area may be incremented by measurement 
coordination device or system 121 A only when the target area is included in an image 

10 stream that is output by production system 102. The embodiment illustrated in 
FIGURE 1 1 may be implemented using a variety of methods. For example, image 
capture and telemetry systems 104A-104N and image measurement systems 104A- 
104N may be implemented using the same or separate hardware or software 
platforms. These systems may also be used to virtually simultaneously capture image 

15 data from the same event at a single venue, or different events at different venues, and 
may be located at the same or separate venues, or collocated in a production control 
unit such as a production truck. 

In FIGURE 12, there is a flowchart depicting the processing of a real-time 
duration calculation module 147. The method may be performed for one or more 

20 target areas included within an image in accordance with the methods of the present 
invention and duration calculations may accordingly be made with physical images 
within target area 504 as well as synthetic images inserted into target area 504. The 
module 147, in step 1500, receives input from the occlusion separator 144 and the 
background/target area reference separator 138. In a model having a plurality of target 

25 areas, each target area may be assigned a unique index (i) for identification thereof. 
The module analyzes final image 400e, or one or more constituent components 
thereof, for inclusion of the target area 1? at step 1502, and if target areaj does not 
appear in the frame, index i is incremented, at step 1504, and compared against a 
FINAL index value, at step 1506, corresponding to the total number of target areas 

30 included within the model. If the FINAL index value has not been reached, the 
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module processing returns to step 1500. If the FINAL index value has been reached at 
step 1506, module 147 processing ends for the current frame and is repeated from step 
1500 for any subsequent frames. 

Confirmation of inclusion of a target areai within the final image 400e at step 
5 1502 causes an increment of an on-air counter associated with target area; at step 
1508. The counter may be maintained within the calculation module 147 or another 
component of the image measurement system 100 and is preferably a digitally stored 
value representing an accumulated duration that the associated target areai is included 
within the image stream. The measured accumulated duration may be made in any 

10 number of units, for example duration of time in seconds, number of frames, number 
of fields, etc. Moreover, the duration unit may be dependent on a particular image 
standard used for outputting the image stream to an audience. For example, numerous 
video standards, such as National Television Standards Committee (NTSC) standards, 
phase alternating line (PAL) standards, and other video standards, respectively define 

1 5 transmission frequencies that define the number of image frames or fields included 
within an image stream per a given unit of time. Duration measurements of target 
areas included in an image stream transmitted to an audience according to any defined 
standards may be made in accordance thereof. Furthermore, the measured duration 
may be weighted by numerous factors including occlusion and foreground parameters 

20 effecting the viewability of an image within target area 504 as described more fully 
hereinbelow. Index i is next incremented at step 1510 and a comparison is then made 
between index i and the FINAL index at step 1512. If the FINAL index value has yet 
to be reached, the module 147 process returns to step 1500. If, however, the FINAL 
index value has been reached, the module processing is completed for the current 

25 frame and will continue at step 1500 for any subsequent frames. 

Accordingly, any target areas included in the model and having a target image 
inserted into the target area in a final image 400e will have an associated counter* 
incremented for that particular final image 400e. Likewise, any target areas included 
in the model and not having an image inserted into the target area but, rather, having 
30 the target area assigned to an area of interest such as signage physically located within 
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the site will have an associated counter^ incremented for the particular final image 
400e that indicates inclusion of the physical sign within the particular final image 
400e of the image stream. Notably, counting of frames in which a target area is 
included in final image 400e may be made in real-time concurrently with processing 
5 of the image. Clearly, however, calculation of the frame number in which a target 
area is included in an image stream may be delayed with respect to the telecast, 
transmission or other processing of the image stream. Once a frame count for each 
target areaj is made according to the process generally described in FIGURE 12, the 
duration of any target image (physical or synthetic) included within target areai may be 

10 calculated as the product of the frame counter; and the frames per second of the image 
stream. Thus, tariffs may be charged to advertisements made in the form of target 
images inserted into the image stream on a duration-basis that accounts for the 
duration the target image/s are included in the image stream. Likewise, tariffs may be 
charged to advertisements made in the form of physical signage located in the site that 

15 are captured in the image stream by calculating the duration a target area assigned to 
the particular area of the site including the physical signage is included in the image 
stream. 

The technique for calculating the duration a target image is included in an 
image stream may be enhanced by accounting for factors that effect the "readability," 

20 or visibility, of a given target image, A more desirable duration measurement may be 
calculated by accounting for, and adjusting, the on-air counter for occlusions of a 
target image. According to the technique described hereinabove with reference to 
FIGURE 12, a target area for a particular number of image frames would be assessed 
the same duration count, and consequently subject to the same tariffs implemented 

25 according to the teachings herein, regardless of whether the target area was wholly 
unobstructed or the target area was obstructed in any various proportion thereof. Such 
an implementation of the invention maybe objectionable to potential advertisers. 

In FIGURE 13, there is illustrated a final image 400e including a first hockey 
player 404 and a second hockey player 406 in a scenario that may occur slightly after 
30 that depicted in FIGURE 10. Notably, the occlusion depicted in FIGURE 10 is no 
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longer present, that is player 404 has moved beyond the target area 504 and target 
image 604 is now in full view and, consequently, the target image 604 will be, in 
whole, included within final image 400e, In comparison, only a portion of the target 
image 604 will be included within final image 400e for the scenario depicted in 
5 FIGURE 10. 

The module 147 may calculate an occlusion parameter (A) to account for 
variations in the obstruction to a target area 504, and the target image 604 included 
therein, and tariffs applied to advertisers for telecast or other forms of viewing of 
target image 604 can be more appropriately levied in view of the overall viewability 

10 of target image 604, An exemplary arrangement for calculating an occlusion 
parameter involves interrogating, by module 147, the occlusion separator 144 for 
information regarding occlusion image 400c. The module 147 may acquire from the 
occlusion separator 144 a pixel count of the occlusion, that is a pixel count of only the 
portion of image 400e of hockey player 404 that is included within target area 504. 

15 Likewise, a pixel count of the target image 802 may be obtained from the 
background/target image separator 140. A simple occlusion parameter may be 
calculated from the pixel count of the occlusion and the pixel count of the target 
image 802 by, for example, generating a ratio of the occlusion image 400c pixel count 
to the target image 802 pixel count. The duration counter* associated with the target 

20 area* may be accordingly adjusted. 

In FIGURE 14, there is a flowchart depicting the processing of a real- duration 
calculation module 147 operable to measure the duration that one or more target 
images are included in an image stream and to reflect variations in occlusions of the 
one or more target images in the calculated duration. The module 147, in step 1700, 

25 receives a final image 400e from the image combiner 146. In a model having a 
plurality of target areas, each target area may be assigned a unique index i for 
identification thereof The model analyzes the frame for inclusion of target area i? at 
step 1702, and if target areai does not appear in the frame, index i is incremented, at 
step 1704, and compared against a FINAL index value, at step 1706, corresponding to 

30 the number of target areas present in the model. If the FINAL index value has not 
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been reached, the module processing returns to step 1700. If the FINAL index value 
has been reached at step 1706, module 147 processing ends for the current frame and 
is repeated from step 1700 for any subsequent frames. 

Confirmation of a target area* at step 1702 results in module 147 obtaining a 
5 pixel count of any occlusion of the target areaj at step 1708. Module 147 next obtains 
a pixel count of the target image; at step 1710. An occlusion parameter A, for 
example a ratio of the occlusion image pixel count to the target imaget pixel count, is 
next calculated by the module 147 at step 1712, The occlusion parameter may 
alternatively be calculated as a ratio of the occlusion image pixel count to the target 

10 areaj pixel count. Other variations of the occlusion parameter are possible. Acounteri 
is then incremented as a function of the occlusion parameter at step 1714. For 
example, a wholly unoccluded target area (or target image) may result in a unity 
increment of the counter with variations in the increment to the counten being made 
by subtracting A from a unity increment of counter^ According to the exemplary 

15 calculation of A, a wholly obstructed target area (or target image) would result in a 
zero increment of i because the pixel count of the occlusion would equal the pixel 
count of the target areai (or target image,). Appropriately, no change in the counter 
value would result even though the counter value processing has been performed. 
Furthermore, the counter will increase inversely to the amount of occlusion of the 

20 target area. 

The indexi is next incremented at step 1716 and a comparison is then made 
between the indexi and the FINAL index at step 1718. If the FINAL index value has 
yet to be reached, the module 147 process returns to step 1700. If, however, the 
FINAL index value has been reached, the module processing is completed for the 
25 current frame and will continue at step 1700 for any subsequent frames. 

Accordingly, any target areas included in the model and having a target image 
therein within a final image 400e will have an associated counter! incremented for that 
particular final image 400e and any final count associated with a target area 
accumulated over an image stream will more accurately reflect the amount of actual 
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viewability of the target image included therein. The duration of any target image 
included within a target area may then be determined as a product of the frame 
counterj and the frames per second of the image stream. Thus, tariffs may be more 
accurately levied on advertisements made in the form of target images included within 
5 target areas of an image stream on a duration-basis. 

The invention as thus described may be further enhanced by accounting for 
camera zoom that effects the viewability of a target image included within a target 
area. In FIGURES 18A and 18B, there is respectively illustrated a final image 400e 
that may be obtained at different camera zooms. Within each final image, a target 

10 area 504 having a target image 604 therein is included in the final image 400e. The 
image 400e depicted in FIGURE 18 may be analyzed according to the teachings 
described hereinabove to determine a duration calculation of target areas included 
within the image stream that includes final image 400e, Assuming equivalence in a 
particular scenario (that is, equivalence of target area occlusion, etc.), a target image 

15 604 that is much less visible (FIGURE 15 A) due to camera zoom but nevertheless 
appearing in image 400e will have a duration counter associated therewith 
incremented equivalently to a target image 604 appearing in the foreground (FIGURE 
15B) of final image 400e according to the teachings hereinabove. Because such an 
implementation of an on-air duration count of a target image may be objectionable to 

20 potential advertisers, the present invention provides a further enhancement for 
modifying a counter associated with a target area, and thus an associated target image, 
according to variations in the camera zoom that effect the viewability of a target 
image. 

The module 147 may calculate a foreground parameter y to account for 
25 variations in camera zoom resulting in differences of the viewability of a target image 
604 within a target area 504 and, accordingly, tariffs applied to advertisers of target 
image 604 can be more appropriately levied in view of the overall viewability of the 
target image 604. An exemplary arrangement for calculating a foreground parameter 
involves interrogating, by module 147, background/target image separator 140 for a 
30 pixel count of a target image 802, or alternatively a pixel count of the target area 504, 
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included in a final image 400e. A simple foreground parameter may then be 
calculated from the pixel count of the target image 802 (or alternatively the target area 
504) and the pixel count of the final image 400e by, for example, generating a ratio of 
the target image 802 (or target area 504) pixel count to the final image 400e pixel 
5 count. Generally, the pixel count of final image 400e is a known and fixed constant. 
The duration calculation associated with target image 802 may be adjusted as a 
function of the ratio of the target image 802 pixel count and the final image 400e pixel 
count to accurately reflect the overall viewability of target image 802, 

In FIGURE 16, there is a flowchart depicting the processing of a real-time 
10 duration calculation module 147. The module 147, in step 1900, receives a final 
image 400e from image combiner 146. As noted above, in a model having a plurality 
of target areas, each target area may be assigned a unique indexi for identification 
thereof The model analyzes the frame for inclusion of target areai, at step 1902, and 
if the target areai does not appear in the frame, indexj is incremented, at step 1904, and 
15 compared against a FINAL index value, at step 1906, corresponding to the number of 
target areas present in the model. If the FINAL index value has not been reached, the 
module processing returns to step 1900. If the FINAL index value has been reached at 
step 1906, module 147 processing ends for the current frame and is repeated from step 
1900 for any subsequent frames. 

20 Confirmation of inclusion of a target areai within image 400e at step 1902 

results in module obtaining a pixel count of the target areai (or target imagei) at step 
1910. Foreground parameter y, for example a ratio of the target image; 802 pixel 
count to the final image 400e pixel count, is next calculated by the module 147 at step 
1912. A counter is then incremented by a function of the foreground parameter at 

25 step 1914, for example by simply incrementing the counter by the computed value of 
y. A target imagei occupying the entirety of final image 400e would result in a unity 
increment of counter according to the exemplary computation of the foreground 
parameter y. Variations in camera zoom resulting in a target image appearing 
increasingly distant in the background would result in a proportional reduction in the 

30 increment to the counter^ Inde^ is next incremented at step 1916 and a 
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comparison is then made between the indexj and the FINAL index at step 191 8. If the 
FINAL index value has yet to be reached, module 147 processing returns to step 1900. 
If, however, the FINAL index value has been reached, the module processing is 
completed for the current frame and will continue at step 1900 for any subsequent 
5 frames. 

Thus, any target areas included in the model and having a target image 
included therein in a final image 400e will have an associated counter incremented for 
that particular final image 400e and any final count associated with a target area 
accumulated over an image stream will accurately reflect variations in the viewability 

10 of the target image resulting in different camera zooms used to produce any number of 
final images 400e. The duration of any target image included within a target area 
included in one or more frames of an image stream may then be determined as a 
product of the frame counter and the frames per second of the image stream. Thus, 
tariffs may be more accurately levied on advertisements made in the form of target 

1 5 images within target areas included in an image stream. 

Preferably, the image measurement system illustrated in FIGURE 1 may have 
communication channels between the director's console 123 and the image calculation 
module 147. These thresholds may be set or defined by an operator of the director's 
console 123. It has been considered that the amount of occlusion of a particular target 

20 area, or target image, may be so great in some instances while the target area or target 
image is still included within a frame of a telecast that viewers of the telecast may fail 
to recognize the particular target image included within the occluded target area. For 
example, a target image may include only a corporate insignia that visibly occupies 
only a small portion of a target area when included in the image stream. If a portion 

25 of the target area is occluded, the target area may itself have a significant portion 
thereof unoccluded while the insignia inserted in the form of a target image is 
unviewable, or unrecognizable, to viewers of the image stream. According to the 
novel techniques above, in such a circumstance the counters associated with the target 
area would still be incremented because a portion of the target area is included with 

30 the associated frame. Any tariffs calculated as a function of the counter would then 
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include at least a portion thereof for telecast frames where the insignia is unviewable 
or unrecognizable. Thresholds may be implemented to facilitate remedying of such a 
situation. An occlusion threshold may be defined such that increments to the counter 
are not allowed when a specified percentage of the target area is occluded, that is the 
5 occlusion threshold disallows increments to the counter if the occlusion parameter is 
greater than the occlusion threshold. If more than one target area is included in the 
site model, occlusion thresholds may be defined for each target area and can be 
implemented independent of one another. Thus, the advertising director could tailor 
tariffs specifically towards particular target images to be included within a given 
10 target area. 

The occlusion threshold may also be implemented as a function of the 
percentage of the target area, or target image, that appears on screen, that is within a 
frame, that is not occluded. For example, during sporting events, camera panning 
occurs often and will result in frames where target areas are only partially included 
15 within a frame. An occlusion parameter distinct from that described above may be 
implemented in these situations. Thus, a different occlusion threshold for disallowing 
increments to a counter associated with a target area may be implemented for target 
areas that are not wholly included within a frame. 

Yet another embodiment of the invention utilizes sub-target areas 504A and 
20 504B within a target area 504. Tariffs may be calculated on individual sub-target 
areas by assigning individual counters to the sub-target areas. Individual and 
independent occlusion parameters and foreground parameters may be calculated for 
each of the sub-target areas 504A and 504B to facilitate additional refinements in the 
duration calculation techniques described herein. The sub-target areas 5 04 A and 
25 504B may, for example, be useful when a target image includes graphics, such as a 
corporate logo, as well as text. Because graphics may be more or less recognizable 
than text, a target image that includes both can have multiple sub-target areas defined 
within a larger target area. One or more target images may be included within target 
area 504 such that the duration calculation of sub-target areas 504A and 504B allows 
30 independent calculation of the duration of the target image or images within the target 
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area 504. Accordingly, independent counters may be respectively associated with the 
sub-target areas. As of consequence, tariffs may be implemented individually for the 
sub-target areas. Likewise, individual occlusion parameters and/or foreground 
parameters may be calculated for each of the sub-target areas 504A and 504B. 
5 Different occlusion thresholds may be set for each of the sub-target areas 504A and 
504B as well. 

It should be understood that the particular technique for calculating an 
occlusion parameter and a foreground parameter and/or assigning thresholds thereto 
that have been described are exemplary only. Variations for accounting for 

10 differences in viewability due to occlusions and variations in zoom of a camera 
capturing an image stream are possible. For example, a percentage of a target area 
that is within an image frame independent of whether or not it is occluded may be 
calculated. The total area of all occlusions to the target area may then be subtracted 
from the area of the complete image frame prior to calculating the percentage of the 

15 target area that is within the image frame. 

Additionally, variations for foreground compensation of a target area may be 
made. The counting of a frame/field including a target area may be dependent upon a 
threshold for the calculated percentage of target area within the overall frame area and 
percentage of screen area. Referring to FIGURE 18, a total area 2200 of image frame, 
20 or frame signal, is designated A s ; a total area of the viewable portion of the image 

frame 2201 is designated A v ; an entire area of the target area 2202 rendered according 
to the telemetry from the camera is designated A T ; an area 2203 of the target area 
visible on-screen is designated A T ; and a total area 2204 of all occluding bodies of 
the target area is designated A T . 6 T is defined as a threshold for the frame area of the 
25 sign for allowing a count increment. Preferably, 9 T is user-settable. 6 T is a 

corresponding threshold in where the area of all occluding bodies is taken into 
consideration. 6 S represents a user-settable threshold for visible target area that does 
not account for occluding bodies as a percentage of screen area. 0 S denotes the 
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threshold for visible target area accounting for occlusions as a percentage of image 
frame area. The following possible tests may then be performed for determining 
whether a counter assigned to a target area is to be incremented: 
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Equations 1-4 allow thresholds to be set that govern increments to counters 
assigned to respective target areas. Allowance/disallowance of increments to a counter 
10 of a target area may also be determined from a logical or weighted combination of 
these equations as desired. 

The present invention contemplates including one or more sub-target areas defined 
within a target area, e.g., an advertiser's logo within a sign (physical or synthetic) 
corresponding to a target area. Referring again to FIGURE 18, the total area 2205 of a 
15 sub-target area within target area 2202 is designated as^4 z ; the area 2206 of the sub- 
target area within the visible image frame is designated A Lg ; the area 2207 of the sub- 
target area occluded by all occluding bodies is designated A L . The following 
thresholds may then be defined: <f> L is a threshold of the image frame area of the sub- 
target area as a percentage of the total sub-target area; fi L is a threshold of the image 

20 frame area less the total area of all occluding bodies as a percentage of the total sub- 
target area; $ s is a threshold of the sub-target area as a percentage of the total visible 

image frame area; and $ s is the threshold relative to the total image frame area less 

any occluded area of the sub-target areas. 
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The following tests for allowing/disallowing increments to a counter assigned to a 
target area may then be defined: 

A 

- J T> ( I>l (5) 
A 

A - A 
A 

5 -r><f>s ( 7 ) 

Ay 

A ~A 

V 

Allowance/disallowance of increments to a counter of a target area may also be 
determined from a logical or weighted combination of equations 5-8. As an added 
level of sophistication, the equations 1-4 could be combined with equations 5-8 to 
10 provide a hybrid metric for allowance/disallowance of counter increments. 

Preferably, an image measurement system 100 including an on-air duration 
calculation module 147 would implement an on-air duration calculation having 
adjustments for both occlusion and foreground variations as described with reference 
to FIGURES 17 and 19 and/or with reference to one or more of equations 1-8. 

15 However, the teachings of the invention, in the most simple form, may be 
implemented according to FIGURE 12 and the accompanying description. As earlier 
alluded to, calculations may be performed on a field, rather than a frame, based 
mechanism. Thus, the described counters for measuring the duration a target area is 
included within an image stream may be performed by analysis of one or more fields 

20 of a frame of the video stream being telecast. 

Other modifications of the invention are possible. For example, the techniques 
described allowing real-time calculations of the duration that one or more target areas 
are included within an image stream also provide a mechanism for ensuring a desired 
duration a synthetic image is included within an image stream is achieved. For 
25 example, an advertiser may contract to have an advertisement included in an image 
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stream, such as a telecast of a sporting event. The present invention allows the 
contract to specify a duration, such as a maximum exposure time, a minimum 
exposure time or a precise exposure time, that an advertisement, or other synthetic 
image, is included in an image stream. The contract may specify a number of units, 
5 such as seconds or frames of an image stream, an advertisement or other image is to 
be included in the image stream. Accordingly, advertising fees levied for inclusion of 
an advertisement in an image stream may be made on a duration basis prior to 
telecast, or other transmission mechanism, of the image stream. This is a notable 
improvement over prior art techniques that require calculation of the duration an 

10 advertisement is included in an image stream to be made after the telecast or other 
transmission of the image stream. Duration-based fees for images included within an 
image stream may not be levied until after telecast of the image stream according to 
techniques known in the art. Additionally, the duration a particular image is included 
within an image stream may not be defined prior to transmission of the telecast 

1 5 according to the prior art techniques. 

A synthetic image may have a desired exposure time defined prior to 
transmission of an image stream that is to have the synthetic image inserted therein 
according to an embodiment of the invention. Assigning a synthetic image to a target 
area for insertion therein and allowing the target area to be dynamically reassigned 

20 during an image stream transmission may facilitate meeting a predefined, desired 
duration an image is to be included in an image stream according to the teachings of 
the invention. For example, during a sporting event telecast, a synthetic image 
assigned for insertion in a target area may have a pre-defined desired exposure 
duration defined prior to transmission of the image stream. Real-time calculation of 

25 the exposure time of the duration the target area having the synthetic image assigned 
thereto may indicate, at some point during the image stream transmission, that the 
target area may not be included within the image stream for the desired duration of the 
advertiser. The synthetic image may be reassigned to another target area that is 
determined to provide a greater likelihood to be included within the image stream for 

30 the original, pre-defined duration associated with the image data. For example, an 
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advertiser may contract to have a synthetic image inserted in a target area for a 
predefined duration of a sports venue such as a right field wall of a baseball field. 
During transmission of the baseball game, analysis of duration measurements, metrics 
or other statistical data collected during capture of the image stream and calculated for 
5 the target area having the image data assigned thereto may indicate that the target area 
will likely be included for a duration that is less than the desired duration indicated 
and agreed upon by the advertiser prior to transmission of the image stream from the 
venue. Accordingly, the image data assigned to the target area may be reassigned to 
another target area. For example, if during transmission of the image stream it is 

10 determined that the actual duration the target area having image data assigned thereto 
is likely to have a duration that is less than a predefined desired duration, the image 
data assigned to the target area may be reassigned to another target area, for example a 
target area in a more visible area of the venue, such as a center field wall, having 
duration calculations indicating a greater exposure duration relative to the target area 

15 having the image data originally assigned thereto. Thus, once the image data is 
reassigned to another target area, the duration the image data is included in the image 
stream may be increased in an attempt to have the image data included within the 
image stream to achieve the predefined, desired duration. Likewise, the image data 
may be reassigned to multiple target areas and reassignment thereof does not preclude 

20 maintaining an assignment of the image data with a target area from which it is 
reassigned, that is the image data may be assigned to a first target area and may be 
reassigned to a second target area upon which the image data originally assigned to 
the first target area will appear in images of the image frame including either the 
second target area or the first and second target area. Reassignment of image data 

25 may be made to any number of target areas in an attempt to include the image data 
assigned thereto within the image stream for a predefined duration. Accordingly, 
variations of the counter mechanisms described herein may be made to facilitate 
duration measurements of image data included within one or more target areas and 
that may be reassigned to other target areas. For example, counters assigned to a 

30 target area may similarly be assigned to particular image data that may be inserted into 



ATTORNEY'S DOCKET 
4904.2-7 



35 



PATENT APPLICATION 



images of an image stream and calculated according to the teachings herein. Thus a 
particular counter associated with image data that is to be inserted into a target area 
may be incremented only when the target area is included in an image of the image 
stream and when the target area has particular image data included therein. The 
5 counter may be reassigned to a different target area upon reassignment of image data. 
Furthermore, increments to the counter may be made upon determination of inclusion 
of any one of a plurality of target areas within an image of the image stream thereby 
facilitating reassignment of image data and duration calculations thereof. 

The foregoing description is made in reference to exemplary embodiments of 
10 the invention. However, an embodiment may be modified or altered without departing 
from the scope of the invention, which scope is defined and limited solely by the 
appended claims. 



